Selection Bias Correction in Supervised Learning with Importance Weight. (L'apprentissage des modèles graphiques probabilistes et la correction de biais sélection)

In the theory of supervised learning, the identical assumption, i.e. the training and the test samples are drawn from the same probability distribution, plays a crucial role. Unfortunately, this essential assumption is often violated in the presence of selection bias. Under such condition, the standard supervised learning frameworks may suffer a significant bias. In this thesis, we use the importance weighting method to address the selection bias problem in supervised learning. We first introduce the supervised learning frameworks and discuss the importance of the identical assumption. We then study the importance weighting framework for the generative and the discriminative learning under a general selection scheme and investigate the potential of Bayesian Network to encode a priori assumptions about the relationships between the variables in study, including the selection variable, and to infer the independence and the conditional independence relationships that allow the selection bias to be corrected. We pay special attention to covariate shift, i.e. a special class of selection bias where the conditional distribution, P (y|x), of the training and of the test data are the same. We propose two methods to improve the importance weighting for covariate shift. We first show that the unweighted model is locally less biased than the weighted one on the low importance instances, and then propose a method that combines them in order to improve the predictive performance in the target domain. Finally, we investigate the relationship between the covariate shift and the missing data problem for data sets with small sample sizes and study a method that uses missing data imputation techniques to correct the covariate shift in some simple but realistic scenarios.